AWS monitoring - definition & overview

Monitoring is the process of collecting, analyzing and using data to track various systems. Amazon Web Services (AWS) monitoring scans your AWS resources and applications, collecting data to ensure everything is operating smoothly and securely. Monitoring your AWS infrastructure helps identify vulnerabilities and issues, predict performance and optimize configurations. This practice relies on various tools and services to collect, analyze and present data insights.

Enterprises increasingly depend on cloud service providers to manage their workloads, data and applications. AWS is the most dominant cloud service provider in the world today, representing 41.5% of all application workloads that exist in the cloud and a 30% market share. AWS offers a comprehensive suite of cloud-based solutions based on the internal infrastructure model that Amazon used to launch its wildly successful international retail and shipping operations. Designed for organizations of all levels, AWS features a versatile range of Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS) product offerings.

Organizations are increasing the number of applications they have deployed in the cloud. As organizations move away from on-premise IT infrastructure and towards hybrid cloud environments, you can expect to see more trending to cloud environments and SaaS integrations

For companies that subscribe to AWS, prioritizing reliability and security means investing in an AWS monitoring solution and AWS security that complements existing AWS tools, satisfies the requirements of the AWS shared responsibility model, and enables best practices for AWS monitoring. As the recipient of the AWS ISV Partner of the Year, Sumo Logic's analytics platform is known for its:

• Ability to drive the reliability and security of AWS environments with real-time insights from its cloud-native platform.

• ROI: Delivering an average of 171% return on investment to our customers.

• Experience supporting cloud migration and business success through SaaS innovation with AWS and other cloud platforms.

• Easy, out-of-the-box AWS integrations to enhance and protect AWS environments.

There are many discussions in the DevOps world about the difference between monitoring and observability. Monitoring, by definition, is the process of collecting, analyzing and using data to track various systems. Whereas, observability leverages all of the data from logs, metrics and traces to help development teams detect and resolve any issues. Observability focuses on understanding the context of all the metrics and your infrastructure's internal state.

In simple terms, monitoring is capturing and displaying data, and observability is understanding system health through inputs and outputs.

>>Watch the Sumo Logic AWS Observability demo.

To learn more, you can click here to read about AWS Observability.

To help meet the cloud reliability and security needs of its customers, AWS offers a range of tools that IT organizations can use to monitor and secure their AWS cloud environments. These tools are available on a pay-per-use basis for AWS subscribers. As a leading partner of AWS, Sumo Logic has integrations with many of these services to keep customers’ AWS-based applications stay reliable and secure.

CloudTrail is a monitoring tool that AWS subscribers can use to track user activity and API usage across the AWS infrastructure. CloudTrail automatically records and stores event logs of actions made in each AWS user account, offering total transparency and visibility into user and resource activity through the infrastructure. Log files include information such as the date and time that a user interacted with AWS, the identity of the user and the IP address where the traffic originated.

Learn more about AWS CloudTrail.

The CloudWatch service was specifically engineered to meet the requirements of DevOps engineers and developers for security and operational monitoring throughout the IT infrastructure. CloudWatch offers many versatile monitoring capabilities, including anomaly detection, automated incident responses, troubleshooting, operational insights and metric visualization. Amazon has marketed AWS Cloudwatch as the easiest way to aggregate metrics from both AWS and your on-premise cloud infrastructure.

You can use our AWS Lambda function to collect Amazon CloudWatch Logs and seamlessly convert CloudWatch logs into a format that is compatible with Sumo Logic.

Sumo Logic uses the Lambda logs API to send platform, function and extension logs to Sumo Logic.

The Certificate Manager tool for AWS simplifies the process of provisioning, managing and deploying SSL/TLS certificates for AWS services. Unlike other AWS services that operate under the pay-per-use model, the AWS certificate manager provides certificates and automates renewals for free — users only pay for the computing power necessary to run the application.

AWS CloudHSM (hardware security module) gives AWS users control over encryption keys and cryptographic operations that are used to codify sensitive data, such as usernames, passwords, credit card numbers and personal information about customers. Effective use of encryption can help IT organizations satisfy corporate, contractual and regulatory requirements associated with collecting or capturing customer data.

With Graviton2, AWS customers with cloud-native workloads can create and run AWS Lambda functions, with improved price and performance. AWS Lambda functions powered by AWS Graviton2 processors offer up to a 34% price-performance improvement over x86-based Lambda functions. To monitor Graviton and Lambda functions, users turn to analyze functions and extension logs. This ensures you can remediate errors and exceptions that run on Graviton.

With Sumo Logic AWS Observability, you get an aggregate view of how all of your AWS Lambda functions are performing.

Want to learn more? Read our guide on AWS Graviton Monitoring.

Amazon Inspector is a security assessment tool for AWS that allows users to perform automated security assessments of applications deployed in AWS environments. AWS inspector detects anomalies or variations from baseline activity or traffic levels, generating alerts that can be investigated from within the service or aggregated into the AWS security hub service along with security data from other services.

Amazon Elastic Kubernetes Service (EKS) is a managed service that helps make it easier to run Kubernetes on AWS, where the majority of Kubernetes workloads live. Through EKS, organizations can run Kubernetes without installing and operating a Kubernetes control plane or worker nodes. Simply put, EKS is a managed containers-as-a-service (CaaS) that drastically simplifies Kubernetes deployments on AWS.

For EKS, CloudWatch offers built-in monitoring. Organizations use CloudWatch to collect and track metrics on AWS cloud resources and the apps that run on AWS. Many organizations rely on CloudWatch to collect and monitor log data, set alerts and alarms and configure workflows to automatically audit and react to changes in AWS environments.

Ready to learn more? Click here for our guide on AWS EKS.

Security Hub provides a centralized hub where AWS subscribers can aggregate data and security alerts from across the entire range of AWS security applications. Users can capture security data from other services, such as Amazon GuardDuty, Amazon Inspector, and Amazon Macie, organize/prioritize those alerts and view the most important information on customized, integrated dashboards.

AWS Spot Instances use spare capacity from services like EC2 or EKS to help you realize cost savings. Since spot instances leverage unused capacity, they can reduce your AWS costs. Spot Instances are a great option for AI and ML-powered applications that require a fast network, massive amounts of memory and high computing capabilities.

Because AWS Spot Instances can be taken away by AWS with a two-minute notice, it becomes highly important to understand interruptions. Click here to learn more about Monitoring AWS Spot Instances.

Example of a Sumo Logic dashboard that tracks the number of interruptions and rebalances recommendations grouped by clusters, instance types, etc.

Amazon Elastic Container Service (or ECS) is a fully managed container orchestration service offered by Amazon AWS. Amazon ECS is being adopted by several best-in-class companies because of its strengths in scalability, reliability and security. The primary use cases for ECS, include managing microservices and handling batch jobs.

Monitoring ECS can be done with several different tools, which include:

• Built-in monitoring capabilities from ECS.

• CloudWatch, to monitor metrics, such as CPU and memory allocation to address scaling demands.

• CloudTrail, to record all ECS API calls and store log files in an S3 bucket. These stored API calls include user session data.

To learn more about ECS, read our guide on What is Amazon ECS.

AWS EC2 (or Elastic Cloud Compute) is chosen for several reasons. One of the main benefits of AWS EC2 is its capability for elastic load balancing. This automatically distributes incoming application traffic across several instances, while also identifying unhealthy instances and rerouting traffic to the healthy versions until restored. The core metrics that are measured in EC2 through AWS monitoring fall into three categories: CPU usage, network activity and disk operations. These are essential for providing basic visibility into your instances’ health and performance.

To learn more about EC2, read our guide on What is Amazon EC2.

Amazon Elastic Block Storage (EBS) is a cloud-based solution that provides persistent block storage volumes for use with Amazon EC2 instances in the AWS cloud.

Monitoring in Amazon EBS is automatic in the AWS console. EC2 compiles datasets on a standard set of metrics, largely storage performance, every five minutes, although the user can opt to increase frequency to every 60 seconds.

Secondary datasets are customizable by the end user and can include information from the available categories, including several CPU credit metrics, dedicated host metrics and traffic mirroring metrics, among others.

Both datasets can be compiled, viewed and analyzed on the convenient Amazon CloudWatch dashboard for a quick reference to application performance.

To learn more about EBS, read our guide on Amazon EBS.

Amazon ELB (Elastic Load Balancing) comes in two offerings:

Classic Load Balancing. This more closely resembles traditional load balancing, but virtual devices replace physical hardware to evenly distribute your incoming requests and ensure a clean, fast user experience.

Application Load Balancing. Application load balancing identifies incoming traffic and directs it to the right resource type. For example, URLs tagged with /API extensions can be routed to the appropriate application resources, while traffic bound for /MOBILE can be directed to resources managing mobile access.

Monitoring AWS ELB involves understanding latencies and optimizing for the user experience. With ELB monitoring tools like Sumo Logic, users can process and render all raw log data into easy-to-understand visuals with deep forensic analysis.

To learn more about ELB, read our guide on AWS ELB.

AWS Lambda is a serverless, event-driven computer service to run your applications in your AWS environment without worrying about underlying infrastructure such as CPU, memory or storage. The service enables you to upload your code — called a Lambda function — and configure it to execute under certain circumstances.

AWS provides vital stats for Lambda in a couple of ways. You can view metrics from your Lambda console, the CloudWatch console, via the Lambda or CloudWatch command line, or even through the CloudWatch API.

Once you start expanding Lambda usage in your applications and even run entire apps in Lambda, you’ll need the power of a more robust monitoring tool.

To learn more, read our guide on AWS Lambda.

DynamoDB is a NoSQL database. An advantage of DynamoDB over other NoSQL databases is that it is hosted and managed by AWS. Users do not need to provision the underlying hardware or concern themselves with adding additional capacity as the database’s storage needs increase. DynamoDB offers high availability and fault tolerance and can be accessed from around the globe as needed.

Monitoring DynamoDB involves collecting data from CloudWatch and CloudTrail and leveraging third-party tools like Sumo Logic. Tools like Sumo Logic provide you with a comprehensive view of health and performance, as well as help to identify correlations between problems and configurations.

Click here to learn more about DynamoDB monitoring.

GuardDuty is an AWS native threat detection service. It constantly monitors your AWS services and aggregates data from CloudTrail, VPC Flow Logs and DNS Logs. There’s no software or additional steps to implement as it is built into AWS. However, using platforms like Sumo Logic you can increase your AWS security posture by integrating with the rest of your security and incident response tools to aggregate and contextualize the data.

Sumo Logic pulls additional log sources and broader context to provide full-stack security visibility into your application, infrastructure, host, and load balancer; it correlates all these events against CrowdStrike’s threat intelligence feeds.

To learn more, read our guide on AWS GuardDuty.

CloudFront is an AWS service that allows businesses to deploy content with low latency and high data transfer speeds. This allows end users to get an immediate response from the website as soon as they search for information.

Monitoring AWS CloudFront means you’ll need to be able to:

• Analyze visitors and content being served by CloudFront.

• Correlate your CloudFront data with internal data to measure the impact of CloudFront as a CDN.

• Create insights into user activity trails.

• Decrease errors and downtime.

• Troubleshoot and improve the quality of service.

To learn more, read our in-depth article on AWS CloudFront.

The AWS Relational Database Service is a program that enables users to create, scale, manage and generally operate collections of interrelated datasets within the Amazon Cloud.

As with other database monitoring systems and procedures, Postgres monitoring for AWS RDS DB instances helps to ensure their reliability, availability and performance. For AWS RDS monitoring, you can easily connect RDS to CloudWatch to export logs, debug multipoint failures and collect data across your AWS services.

Interested in learning more about RDS? Read our guide on AWS RDS monitoring.

AWS Config Rules are a form of AWS monitoring for managing configuration changes in an automated manner.

AWS Config Rules make it easier for you to manage compliance within your environment and stay up-to-date with configuration changes as they are introduced.

Learn more about AWS Config Rules as a monitoring solution.

Virtual Private Cloud (VPC) Flow Logs are an AWS monitoring solution that lets you capture and log data about network traffic in your VPC. VPC Flow logging records information about the IP data going to and from designated network interfaces. This data is then stored in Amazon CloudWatch where it can be retrieved and viewed. VPC Flow logging is critical for security and compliance in your AWS cloud environment.

AWS VPC flow logs help identify latencies, establish performance baselines and understand other issues related to application performance. Additionally, VPC flow logs are instrumental in addressing root cause analysis of critical security issues.

Ready to learn more? View our guide on VPC flow logs.

AWS CodePipeline is a workflow management tool that allows users to construct and manage a process whereby their code can be built, tested and deployed into either a test and/or production environment. Code Pipeline has built-in monitoring tools that ensure the pipeline is halted if there are any problems in the test, deploy, release process. This ensures that preventable bugs don’t make it into production.

Want to learn more? Read our guide on AWS CodePipeline.

Amazon Simple Storage Service, also known as S3, is a highly scalable, fast and durable solution for object-level storage of any data type. Object storage allows users to upload files, videos and documents like you would upload files, videos and documents to popular cloud storage products like Dropbox and Google Drive.

With monitoring tools like Sumo Logic, network and system engineers can help business teams pinpoint where traffic is coming from when peak and slow traffic points occur and what kind of error codes are being reported by your applications.

Logs are ingested by a Sumo Logic collector. By using the Sumo Logics Operational Overview dashboard, you get a high-level overview of all of your web application logs.

Want to learn more? Read our guide on AWS S3.

Shield is a managed Distributed Denial of Service (DDoS) protection service that provides constant monitoring and detection of DDoS attacks initiated by malicious actors. A DDoS attack floods the servers with packets and can trigger service outages that impact revenue. AWS Shield can be combined with Amazon CloudFront and Amazon Route 53 to achieve full protection from Layer 3 and 4 IT infrastructure attacks.

Web Application Firewall (WAF) is a pay-per-use cloud-deployed software tool that lets you define customized rules that protect your web-based application from common and well-known cyber attacks. With this capability, IT organizations can protect their applications from security breaches, maintain application availability and prevent excessive resource consumption that results from malicious traffic.

Despite Amazon's impressive range of security tools for AWS, it is still incumbent on IT organizations to choose the ones that will be most useful and cost-effective while requiring the least amount of administrative and cost overhead. Enterprises that deploy applications in more than one cloud can also benefit from third-party security monitoring solutions that aggregate data from throughout the hybrid cloud environment - not just from AWS.

Sumo Logic customers typically use 40+ AWS services over multiple regions and multiple accounts. Learn how to gain integrated visibility across your whole AWS presence and reduce the noise to find critical issues quickly.

Deploying AWS microservices can be a challenging process. Below, we’ll link you to a complete guide that takes you through a detailed process of deploying a microservice into the AWS ecosystem. In the guide, you’ll learn about the importance of monitoring during a deployment, and once an application is running in production. You’ll find common challenges and how you can leverage Sumo Logic’s expertise and tools to streamline the process and get you back to developing the next feature on your roadmap.

Read the Complete Guide.

Under Amazon's shared responsibility model for cloud monitoring and security, the onus is on AWS subscribers to implement the tools and software necessary to secure their cloud-deployed applications.

Under the model, Amazon assumes responsibility for the security of the cloud while the customer is responsible for securing everything in the cloud.

This means that Amazon controls the security of all components from the host operating system and virtualization layer to the physical security of the building where AWS servers are located. Customers are responsible for implementing and managing client and server-side encryption and network traffic protection. The security of customer data, access management, platforms, applications, the network and the guest operating system is also the responsibility of the customer.

Sumo Logic’s multi-cloud SaaS analytics platform has integrations for major cloud service providers like Google Cloud, Microsoft Azure and AWS, which continue to take hold as cloud adoption grows.

With Sumo Logic's AWS monitoring capability, users benefit from deep integration with the AWS platform and security services. Sumo Logic's log aggregation capabilities, along with machine learning and pattern detection make it easy for enterprise organizations to gain visibility into AWS deployments, manage application performance, maintain cloud security and comply with internal and external standards. If you are already working with AWS or thinking about leveraging it to amplify your business results, consider Sumo Logic for deep insights into your application performance and security.

Learn more about Sumo Logic’s AWS Monitoring Solution.

What tools are available for performance monitoring in AWS environments?

• AWS X-Ray: A service that helps you trace requests as they travel through your AWS application. With X-Ray, you can analyze and debug performance bottlenecks, identify errors, and optimize your application for better performance.

• ManageEngine Applications Manager: A comprehensive application performance monitoring tool that supports monitoring various AWS services, including EC2 instances, RDS databases, and S3 buckets. It provides insights into the performance of your applications running on AWS.

What are the key differences between CloudWatch Logs and CloudTrail for monitoring AWS services?

CloudWatch Logs is dedicated to log management and analysis, while CloudTrail is focused on providing an audit trail and visibility into AWS API activity and changes within your account. Each service plays a distinct role in monitoring and maintaining the security and performance of AWS services.